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United States, Germany, and Panama. During these visits, program 
activities were observed and administrators, teachers, participants, 
graduates, noncommissioned officers, and conaanders were administered 
questionnaires and interviewed. The studies found that all programs 
improve soldiers* basic skills; however, many soldiers graduate from 
basic skills programs without achieving criterion scores on standard 
tests. Soldiers show greater enthusiasm for job-related curricula 
than for materials lacking milite^y content and express positive 
opinions about teachers. Factors common to all programs that may 
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FOREWORD 



The Curriculxam and Evaluation Team of the Instructional Technology Sys- 
tems Technical Area of the U.S. Army Research Institute for the Behavioral 
and Social Sciences performs evaluation and curriculum development applicable 
to military education and training. A major focus of this research is the 
development of information on which the Department of the Army can base de- 
cisions about its Basic Skills Education Program. 

This report sumirarizes three years of evaluation research on the Army's 
Basic Skills Education Program. General findings are that the program im- 
proves the skills in which eligible soldiers are deficient. However, many 
solri-iers graduate from the program without achieving criterion scores on 

' frosts. Common factors that may dilute program effectiveness include 
s' lack of specialized training and experience, the wide range of skill 
levels within classes, and ongoing personnel turbulence. 

This research effort was supported by the Office of the Adjutant General 
of the Army and the Training and Doctrine Command. 




EDGAR M. JOHNSON 
Technical Director 
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THREE YEARS OF EVALUATION OF THE ARMY'S BASIC SKILLS EDUCATION PROGRAM 
EXECUTIVE SUMMARY 



Requirement : 

The Department of the Army has a need for information on which to base 
decisions about the future of its Basic Skills Education Program. 



Proc dures: 

Two major sources were used to compile information about standard, pilot, 
revised, and developing basic skills progreims: 

1. Data files from the Training and Doctrine Command, Enlisted Master 
File, and Defense Manpower Document Center, and 

2. Field visits to Army posts in the continental United States, Germany, 
and Panama. During these visits, program activities were observed and admini- 
strators, teachers, participants, graduates, noncommissioned officers, and 
Commanders \vere given questionnaires and were interviewed. 

Findings : 

All programs improve soldiers' basic skills. However, many soldiers 
graduate from basic skills progreims without achieving criterion scores on 
standard tests. Soldiers show greater enthusiasm for job-related curricula 
than for materials lacking military content, and express positive opinions 
about teachers. Factors common to all programs that may influence their 
effectiveness are teachers' lack of specialized training and experience, the 
wide range of skill levels within classes, and reduced resources and personnel 
turbulence resulting from the low-bid system. 

Utilization of Findings: 

The Department of the Army can use the?^e f .dina^^ 3 . . " f^l.. 
decisions or er^^Trirr th'" Basic SV- ■ 3 "^^^ ^ 
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YEARS OF EVALUATION OF THE ARMY'S BASIC SKILLS EDUCATION 



In 1980, the Army Research Institute was tasked by the Office of the 
Adiutant General of the Army to carry out an evaluation of the Army's Basic 
Skills Education Program. Under contract with the Army Institutes for Re- 
search, this effort has been ongoing since February 1981. During this period, 
a great number of reports and memorandums documenting evaluation activities 
and findings have been produced. The following report has been prepared to 
consolidate and summarize this information. Detailed discussions of evalu- 
ation results and data displays can be found in the reports cited in the text. 



GENERAL 



The Basic Skills Education Program (BSEP) is a part of the Army Continu- 
ing Education System— the primary delivery system for training and educational 
opportunities designed to help soldiers succeed in their careers. The programs 
grouped under the BSEP rubric provide enabling skills in language, literacy, 
arithmetic, computation, and speaking. The underlying rationale for BSEP is 
that some enlistees lack sufficient literacy and/or English-language skills 
to perform their jobs effectively, and that these deficiencies can be remedied 
by BSEP programs . 

BSEP I designates programs in literacy and English as a second language 
(ESL) offered at the training base prior to or concurrent with Initial Entry 
Training (lET). BSEP II progran.3 involve soldiers assigned to permanent 
duty stations after completing lET. All of the programs are monitored by 
Education services Officers (ESC) and instruction is delivered by contract 
teachers. During the period covered by this report, teachers had either 
individual contracts or wer*> employed by institutions contracted to Provide 
instructional services. Eac.i contract institution supplied its own syllabus, 
curriculum materials, lesson plans, and instructional approach to literacy 
training. In the case of ESL programs in both BSEP I and BSEP II, most 
contractors used the American Language Course developed by the Defense 
Language Institute (DLI) . However, adaptations of the course materials to 
the needs of differing groups of soldiers, and the manner in which teachers 
tended to enhance classroom instruction by introducing materials such as 
nuLlitary charts and models, resulted in substantial i'^^^^^'^^^^'^^^^^f ^ 
from class to class. It is important to note, however, that, although BSEP 
delivery methods varied, program goals remained constant; "...to satisfy tne 
skill development and occupational needs of the Army in defense of the 
nation; and increase soldiers' career potential, job satisfaction, and 
educational growth" (Army Regulation 621-5, 1981). 

Potential candidates for BSEP training are soldiers who fail to meet 
the criterion score on the General Technical (GT) subtest of the Armed 
Services Vocational Aptitude Battery (ASVAB) or lo speak English as a 
second language. These soldiers are screened fo. enrollment eligibility 
on the basis of scores on the Test of Adult Basic Education TABE) f^ 
general literacy training and the English Comprehension Level Test (ECLT) 
for language training. Those who fail to meet test criteria are, at the 
discretion of their Commanders, permitted to enroll in the program. 
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Another way to participate in BSEP is by means of command referral. 
That means that soldiers' supervisors judge that they need literacy or 
English-language training in order to succeed in lET or in their duty assign- 
ments. A third way to enroll in BSEP is by means of individual requests 
made by soldiers and approved by their commanders. 

Not all eligible soldiers enroll for BSEP training. In particular. 
Enlisted Reserve and National Guard eligibles commonly find it difficult to 
extend their annual training requirements to accommodate BSEP. Even in the 
case of Active Army eligibles, participation is disapproved when soldiers 
cannot be spared from their jobs. As a result, it is possible to compare, 
on a variety of indices, performance of BSEP graduates with that of soldiers 
eligible for BSEP who did not enroll. 

In addition to evaluating established literacy and ESL programs, our 
task encompassed pilot and revised programs, as well as programs under de- 
velopment. We also carried out a Needs Assessment for a program currently 
being developed, and evaluated materials developed to identify prerequisite 
competencies needed by soldiers for satisfactory performance of job tasks. 
All of these activities are described below. 

The first group of programs to be discussed, BSEP I programs, include 
the BSEP I literacy programs, two special programs developed for specific 
Military Occupational Specialties (MOS) and materials develop**^, to provide a 
basis for production of job-relevant BSEP curricula covering 94 of the Army's 
highest density MOS. 



BSEP I PROGRAMS 

Basic Literacy Programs 



These programs, offered at training bases, provide soldiers whose screen- 
ing tests result in reading levels below fifth grade with training in remedial 
reading, writing, and computing skills. The training usually lasts from 3 to 
6 weeks, and oach Army post selects a competitive contractor who provides 
instructional materials and teachers (Krug, Hahn. s Wise, 1983). 

Our mettiod for evaluating these programs was to analyze data files from 
the Training and Doctrine Command (TRADOC), Enlisted Master File, and the 
Defense Manpower Document Center. These files covered records of both pro- 
gram participants and eligible nonparticipants for the years 1979, 1980, and 
1981. Variables analyzed included test scores (Adult Basic Learning Examina- 
tion, TABE, and ASVAB), rates of attrition, pay grades. Enlisted Efficiency 
Report (EER) scores, and Skill Qualification Test (SQT) scores (Krug & Wise, 
1 SQ2s, ) • 

When participants and eligible nonparticipants were matched for ability 
level (screening test scores) and length of service, the average pay grade 
of participants was higher than that of eligible nonparticipants. In addition, 
attrition rates were slightly Dower for participants than for nonparticipants. 
No significant differences were found between these groups for EER or SQT 
scores. in terms of scores resulting from testing prior to and following 



2 



EKLC 



training, BSEP was found to result in modest gains in grade-level scores in 
reading, vocabulary, spelling, arithmetic computation, and problem solvxng. 

Although overall attrition was lower for BSEP graduates, this advantage 
was demonstrated by soldiers in units. No attrition advantage showed up in 
lET either in Basic Combat Training (BCT) or in Advanced Individual Training 
(AIT). This may indicate that BSEP has a greater impact on unit demands 
than on those of lET. The data showed, however, that BSEP exit test scores 
positively correlated with success in AIT, i.e., twice as many low-scoring 
soldiers attrited as did high-scoring soldiers (Krug, Hahn s Wise, 1983). 



MOS Baseline Skills Progrcuns 



One of the two projects entitled MOS Baseline Skills called for pro- 
duction of materials that would permit the development of curricula to teach 
soldiers competencies needed to perform 94 of the Army's highest density MOS, 
as well as common soldier tasks (CST) . This required TRADOC's contractor to 
analyze MOS and CST, identify relevant prerequisite competencies, cluster 
the competencies common to two or more MOS to reduce redundancy, develop 
curriculum models based on the clusters, and then design specifications for 
the curriculum models. Finally, the contractor was required to provide two 
tests for each MOS and for the CST (Hahn s Edwards, 1983). 

The second MOS Baseline Skills project called for development of func- 
tionally oriented (job-related) curriculums for two Signal School MOS; 05C, 
radio teletype operator, and 31 M, multichannel communications equipment 
operator. We were requested to evaluate these projects while development 
was in process so as to provide guidelines and recommend changes prior to 
production of final materials. 

The following two sections describe the procedures used and the findings 
that resulted from evaluating these efforts . 

Baseline Skills for 94 High-Density MOS and CS T. TRADOC's contractor 
produced a taxonomy of prerequisite competencies needed to perform job tasks 
for 94 MOS and CST. Then TRADOC provided us with a sample of job tasks and 
associated competencies for 30 of the 94 MOS so that we could evaluate the 
taxonomic structure. The sample included combat arms, combat support, and 
combat service support jobs. Selected staff members were provided with ma^or 
and minor category titles and taxonomic terms and were requested to develop 
their own taxonomies by sorting prerequisite-- competencies into categories 
judged to be appropriate. These results were then compared to the taxonomy 
constructed by TRADOC's contractor to determine the extent to which both 
groups of analystr; agreed about the structure. 

Results showed complete agreement between the two groups of analysts on 
22% of the competencies sorted into categories and complete disagreement on 
11% of the competencies. The taxonomic categories were not found to be 
mutually exclusive and they did not all reflect the same level of organiza- 
tion (i.e., there were generic categories and mixed categories, content 
categories and process categories). There was substantial variability 
in the levels of specificity of categories and subcategories. We found 
redxindant major categories and misplaced subcategories. The correlation 
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coefficient between Uie two r-oups of ana" -ts was .68. The contractor pro- 
vided TRADOC with a detailed analysis to guide ongoing development (Hahn & 
Edwards, 198J), 

we were also requested to evaluate sample of the desj.gn specifications 
produ ^ by TRADOC' s contractor tc rovide a basis for curriculum development 
ba:-rl on tlu taxonomy of prerequisic. oompatencies . Our lecommendations in- 
cludea tiK- tnte.iration oX military anguage and materials into the lessons to 
be developc 'lurjiuding indicators where mastery oC lower level skills was 
essential to fornance of higher level skills), the need for specific and 
clc rly vriiio,, learning guidelines, and the need to correct confusing techni- 
cal langi-.^s, inconsistent terminology, and writing errors (stoddart s Hahn, 
1983a). Gvvidelines were provided to TRADOC to assist in the development of 
the MOS Baseline Skills design specifications. 

The final step in this effort will be to evaluate the validation of the 
locator and diagnostic tests produced for each MOS and GST. 

The next section describes evaluation of BSEP II programs and includes a 
Neec-s Assessment for a program under development, the Job skills Education Pro- 
grair ( JSEP) , and evaluation of a pilot program to teach learning strategies. 

BSEP II PROGRAMS 

Basic Literacy Program s 

These programs, offered at unit sites, provide soldiers whose screening 
tests result in reading levels below grade nine with remedial literacy train- 
ing. In addition to the methods already mentioned by which soldiers enroll in 
BSEP— command referral, voluntary requests, and GT scores below criterion— 
BSEP II also enrolls soldiers who fail to meet criterion SQT scores. 

The same data files analyzed for BSEP I were analyzed for BSEP II with 
the same overall results: higher average pay grades, lower attrition rates, 
gains in grade levels, and no significant effects on EER or SQT scores when 
participants were compared to eligible nonp?.rticipants . A notable difference 
between BSEP i and ii graduates was found in grade-level gains. The average 
grade-level gains for BSEP II graduates w-re about twice the magnitude of 
gains achieved by BSIP i graduates. In a..Jition, BSEP II graduates showed 
an average gain in GT scores of 1 7 points. 

We traveled to Germany and Panama to carry out a nvunber of program eval- 
uation tasks, among which was asking Commanders and noncommissioned officers 
(NCOS) about their experiences with and opinions of BSEP n and its graduates 
(Stoddart & Hahn, 1983). 

Interviews with Commanders and Noncommissioned Officers 

The general tendency was that Commanders permitted soldiers to enroll 
in BSEP II as a reward for good performance. Most soldiers enrolled to 
improve their GT scores. Their supervisors reported that the greatest need 
was for BSEP to train soldiers in reading, then in writing, and listening 
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skills. However, supervisors felt that the greatest benefits resulting in 
BSEP training were in improved self-esteem, attitudes, and motivation. Al- 
though job performance benefits were, in supervisors* opinions, secondary, 
these supervisors admitted that their soldiers improved in terms cf motivation 
to perform their jobs, that they required less supervision, and that more 
program graduates performed in the top third of all unit soldiers than they 
did before attending BSEP classes. 

Supervisors had mixed opinions about the way that unit activities are 
disrupted when soldiers attend classes. Although they found these disruptions 
inconvenient, they still thought soldiers gained enough from BSEP to make it 
worthwhile. Overall, the Commanders and NCOs were of the opinion that BSEP II 
trainincr had positive effects on soldiers' performance (Stoddart & riahn, 
1983) . 



Job Skills Educa\J.on Program 

A standardized, f unctionalizpd, computer-based BSEP II program is under 
development and is expected to be fielded in 1986. In order for sponsors and 
developers to be guided by concerns and requirements of field personnel, we 
conducted a JSEP Needs Assessment. The Needs Assessment not only revealed 
field personnel's feelings about the forthcoming program^, but also resulted 
in a substantial body of information about existing literacy programs. Al- 
though these results are summarized also, they are provided in somewhat 
greater length in order to give a more comprehensive picture of the way BSEP 
II functions in the fie3.d. Those questioned and interviewed included sol- 
diers participating in BSEP II, teachers, first- line supervisors and Command- 
ers in the Development and Readiness Command, Forces Command, Health Sciences 
Command, Training and Doctrine Command, Western Command, Germany, and Korea 
(Stoddart & Hahn, 1985). 

Participating Soldiers . Most soldiers enrolled in the BSEP II programs 
that we covered had high school diplcxnas or equivalent certificates. The 
programs in which they enrolled usually used self-paced, modular curricula 
on which soldiers worked independently. The soldiers expressed willingness 
to have course materials presented by computer (JSEP presentation also is 
expected to be self-paced, oiodular, and worked on independently). Partici- 
pants expressed favorable opinions of teachers, satisfaction with existing 
course content, and comfort with the difficulty level of the course mate- 
rials. Generally, soldiers claimed that BSEP II was worth their time and 
effort and also reported a preference for morning, on-duty classes that start 
at the beginning of their duty assignments (in contract to Commanders and 
NCOS who would prefer that classes be scheduled prior to duty assignments)* 
Most soldiers reported having enough out-of -class time to study and some 
felt that the programs should be of longer duration. 

Some differences were found between basic skills needs perceived by 
Commanders and NCOs and those of soldiers enrolled in BSEP II. Soldiers 
had somewhat more confidence in their reading, writing, and listening skills 
than did their supervisors. In reading, soldiers reported that they mainly 
needed to learn rules about hew things work and how to order job steps. 
Writing and listening are skills important to job performance, soldiers 
acknowledged, but they felt confident that they could handle job demands in 



these areas. Soldiers also reported that the most useful skill they could 
learn to enhance job performance involved niemorizir materials. In contrast. 
Commanders and NCOs pointed to soldiers' needs in bo. jlc reading, writing, 
and listening, and emphasized that soldiers need to learn to pay attention 
to details and to complete jobs they have started. The point here is that 
some soldiers may be overconfident of their abilities and harbor inaccurate 
perceptions of their basic skills needs. 

Another finding concerns soldiers" use of military manuals. Soldiers 
reported infrequent use of manuals, although the one they were most likely 
to use was the Soldier's Manual, Their greatest problems with this manual 
were matching pictures, diagrams, and schematics to equipment or terrain, 
and extracting information frcrni charts and graphs, 

Copunanders and NCOs > Supervisors of BSEP II soldiers expressed willing- 
ness to support the programs and release soldiers to attend classes in spite 
of disruptive effects on training schedules. Supervisors felt that the 
programs were worthwhile because they improved soldiers' job performance, but 
reacted positively to tlie alternative of scheduling BSEP training prior to 
duty assignments in order to reduce interference with work schedules. These 
supervisors reported that the durations of the programs are appropriate and 
that classes entailing 4 to 6 hours a day seem reasonable. 

Teachers , Most teachers we contacted had received some specialized 
training to teach BSEP II, Most had held their teaching jobs for only 1 
year or less» indicating a high turnover. Their jobs included making class- 
rocxn presentations, tutoring individual soldiers, administering and scoring 
tests, keeping records, and developing classroom materials* Some of the 
teachers had incorporated Army- re levant materials into their classes* Many 
teachers also acknowledged their lack of detailed knowledge of military sub- 
jects* When asked about the test used to screen soldiers for eligibility 
(TABE), teachers reported it to be generally satisfactory but lacking finer 
diagnostic characteristics at reading grade levels below five. 

Teachers tended to disagree with Commanders and NCOs about command sup- 
port for BSEP II, and pointed to repeated instances when soldiers were with- 
drawn from classes for work assignments* In terms of the proposed JSEP 
program, teachers expressed concern that they would be usurped by cOTtputers 
and that soldiers would be deprived of the personal attention and encourage- 
ment they need. Teachers also expressed some reservations about soldiers' 
ability to operate computer equijHnent* 

Perceived Advantages and Disadvantages of JSEP > A summary of our 
assessment reveals that JSEP has a nxnnber of potential advantages over the 
basic literacy courses currently provided in BSEP II, Since it is to pro- 
vide job- related curricula, JSEP is expected to improve job skills and unit 
morale, JSEP curriculum will be common across BSEP II sites, its teacher 
training will be uniform, and because it will be computer based, it could 
reduce the impact of high teacher turnover, reduce the need for teachers to 
plan instruction, and provide ease of record keeping, curriculiim updating, 
test scoring, and instruction at remote sites. 

Some reservations about the new program include possibilities of equip- 
ment breakdowns, delayed repairs, power failures, fatigue or boredcxn as a 
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result of long periods of work at computer consoles r reduction in interper- 
sonal exchanges between teachers and students, teachers' resistance to being 
replaced by computers/ and a need for teachers to become familiar with a 
broad variety of MOS tasks (Stoddart & Hahn, 1985). 

JSE? Lesson Specifications , We were provided with approximately 10% 
of the estimated total number of lesson specifications to be prepared for 
JSEP, CXir purpose in evaluating them during the process of development was 
to provide guidelines to developers early so that any needed redirection of 
effort could be made prior to project completion, we distributed the speci- 
fications to two staff members with expertise in curriculum development and 
requested that they evaluate them in terms of their suitability for produc- 
tion of curricula for the JSEP program. Then we consolidated their comments 
and recommendations • 

In general terms, the specifications were of a sufficiently high qual- 
ity to provide curriculum designers with ample guidance for producing an 
organized curriculum that would meet learning objectives • However, there 
were a few points that merited consideration. 

First, developers should keep in mind that a totally computer-based 
program is a radical departure from existing presentation modes and could 
require extensive orientation and adaptation on the part of the users. 
In this same context, the specifications made no provision for teacher- 
student interaction. 

Second, the specifications would be more helpful to curriculum develop- 
ers if they included more detailed guidance about the intended complexity of 
instruction to be given to each of the prerequisite competencies that will 
be taught o (These prerequisite competencies are those developed for 94 
high-density MOS and CST discussed above under MOS Baseline Skills.) In 
addition, it is important to be aware that some of the competencies were 
originally assigned to the wrong job tasks. These assignments need to be 
reviewed, and it was also recommended that relationships among competency 
categories and job tasks, test items, and learning objectives be defined 
more explicitly. In some cases, prerequisite competencies could be combined 
(e.g., four competencies associated with outlining) so that a single lesson 
could be designed that incorporated all of them. 

Finally, we recommended that some presentation features be reconsidered 
on the basis of their suitability for an adult population (e.g., interjected 
cartoon figures). Some of the rewards and incentives included in the speci- 
fications also seemed somewhat juvenile and needed to be reevaluated (C. P. 
Hahn, personal communication, October 1983). 



Learning Strategies 

In 1981 a pilot program intended to embed learning strategies in BSEP II 
curriculum content was conducted at Fort Knox. The objective of the pilot 
was to determine if training soldiers in cognitive skills (i.e., strategies 
that they could use to organize, interpret, and remember material to be 
learned) would enhance their acquisition of basic skills. The particular 
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program selected for tryout was an adaptation of Feuerstein's Instrumental 
Enrichment Program (IE) (Russ-Eft & McLaughlin, 1983). 



The IE program consists of 1 4 instruments (exercise booklets) that can 
be used to teach basic cognitive skills such as problem definition, analytical 
thinking, systematic searching, perceptual precision, and learning processes. 
The instruments were designed to be used with culturally deprived adolescents 
and led to increased scores on standard tests. The objective of the program 
is to change passive, dependent cognitive responses to autonomous, independent 
responses. 

The Army tryout. did not accomplish the desired goals. That is, soldiers 
d5.d not demonstrate significantly improved levels of either cognitive skills 
or posttest scores on basic literacy tests. The tryout pinpointed areas Oo. 
possible difficulties in administering the IE program so that future adapta- 
tiions could avoid such pitfalls. Our evaluation provided this information. 

The adapted version of the IE program reduced 2 years of instruction to 6 
weeks (soldiers with higher GT scores) and 12 weeks (soldiers with lower GT 
scores), and included only a sample of the instruments used in the fvll-scale 
IE program. Also, the full-scale program was developed for culturally de- 
prived adolescents, not adult, military personnel. Because the 2-year program 
administered to adolescents was successful, attenuating the program and using 
it with adults may have diluted its effectiveness. Another possible important 
difference is that Feuerstein's posttest data were gathered 5 years after the 
ins true ti or. was given. The soldiers at Fort Knox were tested for program 
effects immediately after the coursework was completed. 

Another factor that may have contributed to the lack of substantive 
effects was inadequate teacher training. Teachers never fully appreciated 
the importance of integrating IE materials into coursework and spending the 
full amount of time called for on each instrument. The teachers expressed 
ongoing concern that they were neglecting basic literacy sxibjects which sol- 
diers needed badly. Furthermore, instructors never really mastered the 
necessary skills for relating individual learning strategies to real world 
situations. All of this implies the teachers' incomplete understanding of 
the nature of these learning strategies. 

An additional circumstance that affected the tryout was termination of 
the teachers' contracts part way through implementation of IE. This event 
constituted a clear morale problem so that, by the end of the cycle, some 
teachers were not using any IE materials at all in their classes. 

It is important to note that the soldiers who were provided with IE 
instruction demonstrated gains in test scores just as great as those of 
soldiers in classes in which IE was not included. E\irthermore, it has not 
been determined whether strategies taught during the Fort Knox tryout will 
have long-term effects on these soldiers' learning abilities. Given all of 
the above circumstances, it seems reasonable to conclude that the IE did not 
receive an adequate tryout. 

The final section summarizing our evaluation activities covers ESL 
programs and includes a discussion of the ECLT, 11B Language Skills, baseline 
programs, pilot prograuns, and the new f unctionalized program. 

8 
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ENGLISK-AS-A-SECOND- LANGUAGE PROGRAMS 

The Army provides ESL training both at training bases as part of BSEP I 
and at unit sites as part of BSEP II. Soldiers are screened for eligibility 
by means of the English Comprehension Level Test and those with scores lower 
than 70 (range 0 to 100) are considered in need of English-language instruc- 
tion* Most enrolled soldiers are members of the Active Army. Enlisted Re- 
serve pnd National Guard members tend to reject the offer of ESL training. 

Although soldiers needing Englislv-language training constitute only a 
--.mall part of the enlisted force (approximately 5%), they receive much at- 
tention. One reason for this special focus is that limited-English-speaking 
(LES) soldiers, most of whom are insular Puerto Rican, tend to be a well- 
educated and highly motivated group who, equipped with adequate English- 
language skills, become an asset to the military. An additional reason is 
that the Army is confronting a shrinking recruitment pool during the 1980s 
and anticipates admitting a growing proportion of Hispanic recruits. As 
a result, the Army has supported pilot ESL programs as we.1.1 as development of 
a f unctionalized curriculum to enable LES soldiers to succeed in training 
and job performance* 

English Comprehension Level Test 

The ECLT has been used as the index of English-language proficiency at 
DLI for more than 20 years. However, since the test requires only reading 
and listening, some questions arose about its value when used to test sol- 
diers with marked deficiencies in spoken English. As a result, we developed 
an individually administered test called the Oral Proficiency Test that was 
based on a test provided by DLI and that required both speaking (production 
including vocabulary, grammar, fluency, and pronunciation) and understanding 
spoken English (comprehension). We used this test with 86 participants in 
a 3-month pilot ESL program provided by DLI and in 6-week BSEP I programs. 

Correlations between Oral Proficiency scores and ECLT scores were high 
(.70 to .89) with the exception of one sxibtest — pronunciation. Soldiers 
who had received only 6 weeks of ESL training showed no gains in scores 
on this subtest. A high correlation was also found between ECLT scores 
and ratings by BCT and AIT instructors of soldiers' Englirh-language pro- 
ficiency as well as with attrition in the training base (Harman, Oxford- 
Carpenter, & Redish, 1983). 

MOS 11B Language Skills 

During the initial stage of our evaluation activities, we conducted 
a needs assessment involving limited English-speaking and native English- 
speaking soldiers who were participating in AIT for MOS 11B at Fort Benning. 
We developed a test using Job Language Performance Requirements for MOS 11B 
from an analysis produced by DLI. The outcome was that, on the average, LES 
soldiers missed more than 25% of the test items and those who had ECLT scores 
below 70 missed approximately 50%. These results contrast with those of na- 
tive English-speedcing soldiers who missed about 5% of the test items. A 
small number of soldiers participated in this assessment so that the findings 
must be considered indicative rather than definitive (Holland, 1982). 
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Six~Week English-as-a-Second-Language Programs 

Six-week programs using the American Language Course developed and used 
by DLI were part of BSEP at the beginning of our evaluation efforts, We 
gathered data on these programs not only to assess program effectiveness, but 
also to serve as baseline information for comparing pilot {3-month and 6-month) 
and revised { f unctionalized ) programs, we visited seven TRADOC (BSEP I) sites 
during 1981 and 1982 to administer questionnaires and interview Education Ser- 
vices Offices' staff members^ teachers, and participants and their supervisors. 

Most programs used the American Language Course as the curriculum and 
supplemented it with military information (e.g.. Training Extension Course 
tapes, BCT charts. Soldier's and Field Manuals, etc). Classes tended to 
include soldiers with wide-ranging language deficiencies. This was a r<^sult 
of the open-entry system that permitted soldiers to join classes in progress 
as they reported to the post and were screened for eligibility. In addition, 
enrollments fluctuated and, when enrollments were low, it was not cost effec- 
tive to separate soldiers on the basis of language-deficiency levels. 

Instruction for most programs was provided by contractors. In some in- 
stances, ESOs hired teachers by means of nonpersonal services contracts. In 
both cases, lowest bidding contractors or lowest bidding teachers had to be 
selected. This turbulence could occur on a yearly basis. The result was 
high teacher turnover and low morale as salaries declined from year to year. 
It also required repeated warmup periods and reduced services as new con- 
tractors took over direction of the programs from year to year. 

Soldiers enrolled in the programs had widely distributed ECLT scores, 
although more scores were distributed in the low (0-29) and middle (30-49) 
ranges than in the high range (50-69). Eligible soldiers who did not enroll 
in BSEP for Knglish-language training tended to have ECLT scores at the high 
end of the distribution. This was, in part, because those who did not enroll 
when the initial opportunity was available and who had pronounced deficiencies 
tended to enroll after they demonstrated communications problems. All of the 
soldiers interviewed indicated prior knowledge of English — most frequently in 
school settings in which they were taught to read and write English but had 
little experience with English conversation (Holland, Rosenbaum, Stoddart, 
Redish, Harman, & Oxford-Carpenter, 1984). 

Soldiers tended to feel positively about the ESL programs and their 
teachers. At posts where participants were billeted with English-speaking 
soldiers, they had opportxinities to practice what they learned in classes. 
However, many participants were assigned to ESL companies, and most ESL 
soldiers we interviewed said that the classroom instruction included too 
little English conversation. Many also pointed to a need for more time to 
study outside of classes. About half of the participants felt that the 
6-week program was too brief. 

Soldiers who graduated from the program improved on ECLT scores by an 
average of two points per week. Clearly, those who entered the program with 
scores at the lower end of the distribution failed to meet the criterion 
score of 70 during the 6-week period. We found positive correlations between 
ECLT exit scores and AIT success, attrition, and recycling, and these effects 
were more pronounced for soldiers whose exit scores were above 50. 
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Pilot English-as -a-Second-Language Programs 



A G-month ESL program was conducted at DLI during the fall of 1980 and 
the winter of 1981, The program was ending at the time that our evaluation 
activities began. However, we visited DLI to obtain data gathered during 
the 6-month period and to interview teachers and program administrators. 
Of particular interest was a group of 200 soldiers selected as eligible for 
ESL participation but who were not enrolled in the program. These soldiers 
served as a control group for measuring program effects. Both the enrolled 
and eligible nonenrolled soldiers volunteered either to participate in the 
6-month program or to go directly to BCT. 

During the sumnmer and fall of 1981, a 3-month ESL program was provided 
at DLI. We gathered interview, questionnaire, and testing data on this pilot 
program also. 

Language acquisition in both pilot programs tended to be the same orderly 
predictable phenomenon that was demonstrated in 6-week ESL programs. Partici- 
pants tended to gain two LCLT points per week. In contrast, the group of sol- 
diers who were screened as eligible but who went directly to basic training 
sites showed average gains of only one-half ECLT point per week. These sol- 
diers were retested between 6 and 9 mbnthcJ following initial screening and 
some of them had been enrolled in 6-week ESL programs at training bases. In 
addition, attrition was significantly greater for control group soldiers than 
for pilot program graduates. Another finding common to the pilot programs 
and standard G-week programs was that soldiers felt that instruction should 
include more English conversation and less reading. 

We followed as many program graduates as we could into BCT and AIT. We 
are still analyzing data on long-range effects and will report our results 
later this year. Early indications are that there is a high correspondence 
between soldiers' final ECLT scores and ratings of their English proficiency 
by drill sergeants and AIT instructors. There was a distinction between sol- 
diers who scored below ECLT 50 and those who had exit scores of 50 or mc>re 
in terms of their supervisors' ratings. Program graduates reported that 
their greatest language deficiency continued to be speaking skills. However, 
most said that the programs at DLI had equipped them with sufficient English- 
language skills to complete BCT and AIT (Stoddart, 1982). 

Functionalized English-as -a-Second -Language Program 

In July 1982 TRADOC implemented a job-related ESL curriculum at eight 
sites. DLI personnel developed the curriculum which was designed to provide 
soldiers with 6 weeks of training in English-language skills as well as with a 
vocabulary that would facilitate their progress through basic training. The 
curriculum was developed for soldiers with entry ECLT scores from 50 to 69. 
Our evaluation of this program started in August 1983 and is continuing at 
this writing. The findings detailed below represent information gathered to 
date. Whether any changes in present trends will be documented by the end 
of the evaluation effort remains to be seen (Rosenbaum & Stoddart, 1985). 

Although the functionalized ESL program curriculum was in use at all the 
sites we visited, its presentation varied as a function of supplementary ma- 
terials added by teachers (Training Extension Course tapes, SMART books, etc.) 
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and fluctuating enrollment patterns. At a few posts, program directors decided 
to adhere strictly to curriculum materials and made no attempt to introduce 
ancillary materials. 

The distribution of ECLT entry scores for soldiers participating in 
the job-related program has shifted markedly from a majority with low- and 
mid-level scores to a majority with mid- and high-level scores. Nonetheless, 
more thau 50% of the entry scores are still below 50, and this means that 
these soldiers are not part of the target population for whom the curriculum 
was developed. Participants with lower entry scores spend the first 2 weeks 
of instruction on basic English grammar and vocabulary before advancing to 
the f unctionalized curriculum. Scores on the end-of -course test developed 
to accompany the curriculum reflect the need for extra instructional attention 
on the part of lower scoring soldiers. Participants in the target population 
are more likely to pass this test than are those who enter the program with 
ECLT scores below 50 (2/3 vs. 1/3). Overall ECLT gains, however, averaged 
just about two points per week just as they did in other programs. This 
similarity must be viewed in the context of a military-based curriculum. 
The ECLT includes no military content, so that comparable achievements on 
the part of f \inctionalized program graduates represent a somewhat higher 
level of accomplishment. 

In terms of magnitude of ECLT gains, the data are a bit different from 
those gathered on other ESL programs. The nontarget population (ECLT scores 
below 50) showed substantially greater gains than did participants whose entry 
scores were 50 or higher. This may be because these soldiers spend the ini- 
tial period of instruction studying fundamental language skills. 

It is important to note that the change introduced in the ESL programs 
consisted only of a new curriculum. All the other features of the standard 
ESL programs that may have limited their effectiveness — the low-bid system, 
the teachers without prior ESL experience, the classes incorporating wide- 
ranging skill levels, the soldiers assigned to ESL companies, and the lack 
of emphasis on English conversation — must be assumed to have the same impact 
on the job-related program. However, participating soldiers expressed very 
positive opinions about the program and the teachers and reported that the 
program provided them with enough English-language skills to succeed in 
training (Rosenbaum & Stoddart, 1985). 



SUMMARY 

A general overview of BSEP reveals that it improves soldiers' basic 
skills and that the improvement is greater for soldiers whose entry scores 
are very low. Those who benefit most directly from BSEP, participating sol- 
diers and their supervisors, acknowledge that the program is worthwhile and 
point to enhanced motivation and self-esteem. However, it is important to 
note that a substantial number of soldiers leave the program without meeting 
criterion standards. These soldiers enter basic training or return to their 
unit assignments regardless of performance in BSEP, and this circumstance can 
lead to adverse criticism of program achievements as well as to dissatis- 
faction on the part of Commanders. 
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An equally important point is that the Army is continually trying to 
update and improve BSEP to ensure that it more closely meets program goals. 
The variety of pilot and revised programs discussed in this report — e.g.. 
Instrumental Enrichment, Job Skills Education, extended-duration ESL and 
functionalized ESL programs — are just a few examples of this ongoing con- 
cern. In line with this interest, the final task of our evaluation will be 
to develop and test a Quality Control System that will provide continuous 
information about the status and effectiveness of all basic skills programs 
throughout the Army, 
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